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Modeling and Simulation of the Data 
Communication Network at the ASRM Facility 


Abstract 

This paper describes the modeling and simulation of the communication 
network for the NASA Advanced Solid Rocket Motor (ASRM) facility under 
construction at Yellow Creek near Iuka, Mississippi. Manufacturing, testing, and 
operations at the ASRM site will be performed in different buildings scattered over a 
1800 acre site. These buildings are interconnected through a Local Area Network 
(LAN), which will contain one logical Fiber Distributed Data Interface (FDDI) ring 
acting as a backbone for the whole complex. The network contains approximately 
700 multi-vendor workstations, 22 multi-vendor workcells, and 3 VAX clusters 
interconnected via Ethernet and FDDI. The different devices produce appreciably 
different traffic patterns, each pattern will be highly variable, and some patterns 
will be very bursty. Most traffic is between the VAX clusters and the other devices. 
Comdisco’s Block Oriented Network Simulator (BONeS) has been used for network 
l simulation. The two primary evaluation parameters used to judge the expected 
1 network performance are throughput and delay. 

1.0. Introduction 

The Advanced Solid Rocket Motor (ASRM) facility at Yellow Creek near Iuka, 
Mississippi is part of a National Aeronautics and Space Administration (NASA) 
program to substantially improve the flight safety, reliability, productivity, and 
performance of the space shuttle’s solid rocket motors. The facility is 
government-owned but contractor-operated. Lockheed Missiles and Space Company 
Inc. (LMSC) is the prime contractor. The operation of the facility is directed by the 
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subcontractor Aerojet ASRM division (AAD); RUST International Corporation (RUST) 
is responsible for the engineering and construction of the facility [1]. 

The main aim of this paper is to present the overall communication network 
structure for the ASRM facility and describe the modeling and simulation of the 
network. The section on ‘ASRM Communication Network Structure’ concentrates on 
the network connectivity, the cabling, and the protocols. That section also explains the 
flow of data in the network. The section on ‘BONeS Modeling* gives an overview of the 
BONeS simulator and describes the different BONeS models developed to simulate the 
ASRM environment. The ‘Analysis and Results’ section at the end of the paper 
comments on the network expectations and the network evaluation parameters. The 
section also summarizes the various plots of Mean Delay and Throughput versus 
Traffic Intensity obtained using BONeS and some important conclusions are drawn. 

2.0. ASRM Communication Network Structure 

2.1. Main Computing Center 

Building 1000 (B_1000) provides an efficient means to plan, control, and 
document the manufacture of solid rocket motors for the ASRM project. All the 
workstations and workcells communicate only with the Operational Information 
System (OIS), the Business Information System (BIS), and the Area Supervisory 
Computers (ASCs) in B_1000; there is no peer-to-peer communication. B_1000 also 
provides a link between the business functions and the manufacturing functions of the 
facility. The interconnection between the devices in B_1000 is shown in Figure 1. 

The OIS is a VAX cluster consisting of two VAX 6000 computers, each with one 
FDDI adapter. In addition to this OIS VAX cluster, there are two VAX 4000 computers, 
each with one Ethernet adapter. The BIS is a VAX cluster consisting of one VAX 6310 
and two VAX 6420 computers. Each of the ASCs are VAX 4400 with an Ethernet 
adapter. 
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Figure 1 Inside Building 1000 


B_ 1000 has a Gandalf terminal server. The Gandalf terminal server is a large 
terminal server with a multitude of RS-232 ports. The Gandalf can support 12 
separate Ethernet channels. It is the only terminal server throughout the campus. 

The OIS, the BIS, the ASCs, and the Gandalf terminal server in B_1000 are 
connected to the outside network complex by Cabletron Multi Media Access Centers 
(MMACs), the intelligent hubs. There is a fiber connection between the OIS and the 
Cabletron hubs. There is a copper connection between the BIS and its Cabletron hub, 
and between the ASCs, and their Cabletron hub. 
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In addition to the devices already mentioned, B_1000 has nine more Cabletron 
hubs distributed in two switch rooms for the BIS devices. The BIS devices includes 32 
printers, 25 CAD workstations, and 400 Macintosh computers connected to the BIS 
hub by 10BASET; 50 PCs connected to the BIS hub by 10BASET; 31 Engineering 
workstations on 10BASE2; and 289 dumb terminals connected via Asynchronous Data 
Interface to the Gandalf. The connections of the BIS devices is as shown in Figure 2. 



Figure 2 BIS Hub Connections 
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2.2. Network Cabling at the ASRM Site 


At the ASRM site all the outdoor cabling and much of the indoor cabling is 
optical fiber. Thin— wire coax, thick-wire coax, and twisted-pair are also used in 
B_1000. 

2.2.1. Outdoor Cabling at the ASRM Site 

All the outdoor cabling is optical fiber. All optical fibers used are 62.5 / 125 
micron multimode optical fibers. Every FDDI hub has at least three redundant paths, 
viz. Channel A and Channel B of FDDI and a 10BASE-FL backup. Also every FDDI 
hub has two redundant dual rings. 


Link 

Distance 

(feet) 

Number of 
Workstations 
on the link. 

Number of 
Workcells 
on the link. 

Type of the 
Link 

Link # 1 : (2029) 

6700 

29 

04 

Intensive and 
Non-In tensive 

Link # 2 : (2031) 

4650 

15 

00 

Intensive 

Link # 3 : (1016) 

1450 

27 

16 

Intensive 

Link # 4 : (1000) 

00 

(19+73) 

+ BIS Devices. 

00 

Non-Intensive 

Link # 5 : (2066) 

3550 

09 

00 

Non-Intensive 

Link # 6 : (2030) 

5000 

12 

06 

Intensive and 
Non-Intensive 

Link # 7 : (1012) 

950 

14 

00 

Non-Intensive 


Table 1: Distances of each hub from B_1000 
The manufacturing intensive buildings have two FDDI data paths from B_1000 
with automatic switch-over. One data path is buried, while the other is aerial. 
Buildings 1016, 2029, 2030, and 2031 are the manufacturing intensive buildings; each 
has a hub directly connected to a hub in B_1000. Buildings 2060 and 2076 are 
connected to the hub in building 2029. Each hub receives two pairs of fibers from the 
outside cable plant. All workstations and workcell devices will receive two fibers each 
from the respective hubs. 
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All the manufacturing non-intensive buildings in the complex receive two fibers 
for their hub via the outside cable plant. All the workstations inside the buildings get 
two fibers each from the respective hub. 



2.2,2. Indoor Cabling at the ASRM Site 

For the indoor cabling in the manufacturing intensive buildings, the 
10BASE-FL protocol is used, mainly because it allows lower light levels and 16 
redundant data paths [7]. 
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2.3. Cabletron Devices 


At the ASRM site all the Cabletron MMACs will be MMAC-8FNBs, which 
support up to seven Media Interface Modules (MIM). The first slot in the MMAC will 
be the EMME multichannel management / bridge module. The Cabletron hubs 
provide the necessary security, routing, and redundancy [7]. The different MIMs used 
in the network at the ASRM are listed in Table 2. 


Name of the 
Card 

Type of the 
Card 

Protocol 

Number of 
ports 

Comments 

EMME 

Ethernet Bridge 

Ethernet 

4 ports 

Used as a man- 
agement module 
in all the MMACS 

FDMMIM 

FDDI to Ethernet 
Bridge 


8 ports 

Connects 10 Mbps 
Ethernet to 100 
Mbps FDDI 

MT8-MIM 

DELNI Card 

— 

8 ports 

AUI Transceiver 

FORMIM— 22 

10BASE-FL card 

Ethernet 

12 ports 

Provides connec- 
tivity for 12 
ethemet channels 

TPRMIM— 36 

10BASE-T Card 

Ethernet 

24 ports 

Provides connec- 
tivity for 24 
ethemet channels 

CXRMIM 

DEMPR Card 

Ethernet 

12 ports 

Provides connec- 
tivity for 12 
ethemet channels 

GX-M 

GatorStar Card 

LocalTalk 

24 ports 

24 port LocalTalk 
repeater with a 
LocalTalk to 
Ethemet router 

FDMMIM-04 

FDDI Concentra- 
tor 

FDDI 

4 ports 

Provides 4 con- 
centrator ports 
FDDI connections 


Table 2: Cabletron MIMs used at ASRM 
2.4. Protocols used at the ASRM Site 

For the communication network at the ASRM site, two protocols are specified, 
FDDI and CSMA/CD. All the manufacturing intensive buildings are connected to 
B_1000 by links with FDDI protocol, and all the manufacturing non-intensive 
buildings are connected to B_1000 by links with 10BASE-FL protocol (i.e. CSMA/CD 
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on optical fiber). The protocol inside the manufacturing intensive buildings and the 
manufacturing non-intensive buildings is 10BASE-FL. 

2.5. Data Flow over the Network 

2.5.1. Manufacturing Intensive Buildings 

The manufacturing intensive buildings have FDDI data paths from B_1000. 
The manufacturing intensive buildings viz. 1016, 2029, 2030, and 2031 each has a hub 
directly connected to an OIS hub in B_1000. The workstations in these buildings 
communicate with the OIS 6000 computers. The workcells in these buildings 
communicate via the OIS hub with the two ASCs. 

2.5.2. Manufacturing Non-intensive Buildings 

The manufacturing non-intensive buildings 1001, 1002, 1010, 1025, and 1045 
have 10BASE-FL data paths from B_1000. All other manufacturing non-intensive 
buildings are connected to B_1000 through the nearest manufacturing non-intensive 
hub in buildings 2029, 2066, and 2030. The workstations in all the manufacturing 
non-intensive buildings communicate via the BIS hub with the two OIS 4000 
computers. 

2 .6 , Res ear ch Ob jec tive 

The main objective of the research is to simulate and analyze the network to 
determine its performance under different load conditions. Comdisco’s Block Oriented 
Network Simulator (BONeS) is used to evaluate the performance of the network with 
the given topology and protocols. The two primary evaluation parameters that are 
used to judge the network performance are the throughput and the delay. The aim of 
the simulations is to look into the loading of the OIS, the BIS, the ASCs, and the 
network links due to the traffic generated by the workstations and the workcells over 
the entire site. 
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3.0. BQNeS Modeling 


3.1. Different Methods of Network Modeling T41 

The network modeling can be done by several different means, each having its 
advantages and disadvantages. The first method is by developing a mathematical 
model of the network, normally using queueing theory. This model can then be used 
to provide results of the performance of the network. Due to the simplifying 
assumptions required to use with this type of modeling, it is often not the best 
possible model and often is not feasible. 

The second approach to analyze the performance of a network is to actually 
build the network. Although this approach provides very good results, it is normally 
very expensive, both in time and resources. 

The third approach is that of computer simulation. Using a computer 
simulation the user can model the network to as close to reality as desired. This 
approach is less expensive than building the network; however, the major 
disadvantage of this approach is insuring that the simulation model accurately 
models the real-world network. 

3.2. BQNeS Simulator T61 

The Block Oriented Network Simulator (BONeS) provides an interactive 
graphical environment for simulation-based analysis and design of a broad range of 
communication networks. In the BONeS environment, the network model is specified 
in terms of the network topology, traffic, packet and message (data) structures, and 
protocol functions. The user constructs the network graphically and hierarchically 
using the building blocks from the BONeS model library. The user can also write the 
components of a model in C and incorporate them into the BONeS modeling 
environment. BONeS translates the network model into a C program, executes an 
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event-driven Monte Carlo simulation, computes statistical measures of network 
performance, and displays the results graphically. 

3.3. Network Modeling using BONeS 

Simulation model design consists of three elements : data structures, modules, 
and a system. The modules in BONeS control the flow of the data structures. Many 
basic modules are provided with BONeS, such as decision nodes, random traffic 
generators, and fixed delays. These can be combined to form new modules to meet the 
specific needs of the simulation [6]. 

BONeS is an event driven simulation. All events except traffic generators are 
triggered by a previous event called a Trigger. If a block is not triggered then there will 
be no output. Thus when building a model using the provided blocks, race conditions 
such as parallel inputs must be avoided. Instead, blocks should be cascaded to prevent 
the race conditions. 

The different nodes that are constructed in this simulation are workstations and 
workcells. Models of other nodes, such as CSMA/CD nodes, FDDI nodes, and bridges 
are included in the BONeS library. The default parameters of the CSMA/CD nodes are 
set to the IEEE 802.3 CSMA/CD standard. The packet size is set to the smallest 
possible size of 64 bytes for a worst case analysis. If the packet size is small the 
transmission time will be small with respect to the propagation delay, and more 
collisions will occur [5]. 

A Cabletron MIM is an add-on card that can be plugged into a slot of a MMAC. 
Each MIM has ports on its faceplate to support cabling. Each MIM functions uniquely 
as a repeater or as a bridge. The model of the repeater MIM passes all frames that are 
received on the receive port to all transmitting ports with delay. The delay information 
for each MIM was obtained from [7]. 
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Figure 4 BONeS Model of ASRM Network 

A traffic source model has been developed to model a workstation sending a 
block of data such as a text or graphics screen. The traffic source model sends a set 
number of packets at an inter-arrival rate set by the user, in this case a Poisson 
distribution because traffic on a LAN tends to have a Poisson distribution. 

3.4. Probes Setting and the Simulation Iterations 

Several probes were placed throughout the network to gather statistics during 
the simulation. The mean delay per packet, received throughput, transmitted 
throughput, and the number of completed packets were collected for each separate 
link. Each iteration interval was divided into ten batches in order to collect these 
statistics. 

These statistics were collected by placing a Generic Probe on the Media Access 
Control (MAC) Statistics module in each link. The MAC Statistics module was used 
to measure the delay and throughput for all CSMA/CD workstation models on one link. 
All CSMA/CD workstation model MAC instances share the same memory. Therefore, 
all the CSMA/CD workstation models write the delay and throughput into one memory. 
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Figure 5 BONeS Model of Building 1016 

These statistics are then made available to the Post Processor in BONeS by 
placing a Generic Probe on the MAC Statistics Compute module. 

The traffic intensity per node was varied from 10 KBps to 90 Kbps at twelve 
points during the simulation. The traffic intensity was varied with an exponential 
function to show the knees of the curves. The simulation time per iteration was set to 
one second. The actual computer time to do the simulation was approximately 30 hours 
on a moderately loaded Sun 600/MP with 128 Megabytes of memory. 
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4.0. Analysis and Results 


4.1. Network Expectations 

The network should be reliable and have redundant links since the control of the 
manufacturing will be accomplished over the LAN. Large amounts of data will be 
required due to the extensive monitoring and documentation required for the 
manufacture of the solid rocket motors in the Space Shuttle program [2]. 

4.2. Network Evaluati on Parameters 

The two primary evaluation parameters used to judge the network performance 
are throughput and delay. The throughput is the effective bit rate of the system in bits 
per second (bps). It does not include the overhead bits used by the protocol or the 
packets that have to be re-transmitted. The delay in a LAN is determined by the mean 
delay per packet [3]. 

The delay in a LAN is mostly caused by the following three factors: the 
propagation delay, the delay in a transceiver, and the queuing delay in a bridge. Also 
the user response time is important. These delays are modeled using several different 
techniques in the simulation. 

4.2.1. Propagation Delay 

The propagation delay of the light signal traveling down the fiber is modeled by 
using a fixed delay model provided by BONeS. The link delay is calculated by dividing 
the distance by the speed at which the light travels down the fiber (0.67 times the speed 
of light) [2]. 

4.2.2. Delay in a Transceiver 

The worst case collision detection time of a particular CSMA/CD commercial 
transceiver was found to be 900 nanoseconds [8]. The worst case packet delay of a 
particular commercial optical hub was found to be 630 nanoseconds [8]. This delay is 
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caused by the optical-to-electrical and electrical-to-optical conversion. These delays 
are also modeled by a fixed delay in the simulation [2]. 

4.2.3. Queue Delay in a Bridge 

In the simulation model each of the CSMA/CD networks is connected to the 
FDDI backbone by a bridge. Abridge converts the CSMA/CD packet to an FDDI packet 
and buffers the incoming packets until they are serviced. If a packet enters the bridge 
and the queue is full, the packet is discarded. If the queue is large but not full, then 
the packet will be delayed. 

4.3. Mean Delay and Throughput Plots 

The mean delay per packet versus the offered traffic intensity plots are shown 
in figures 7, 9, and 11. The throughput versus the offered traffic intensity plots are 
shown in figures 6, 8, and 10. 

4.4 . Ob servation s an d Res ult s fro mjhe. Pl o ts 

The statistics collected with the probes during the simulation are plotted to 
judge the performance of the network. The mean delay per packet and throughput are 
plotted versus the offered traffic intensity. These plots are created using the Post 
Processor in BONeS and show the performance of the network at each iteration of the 
traffic intensity during the simulation. 

4.4.1. Mean Delay Plots 

The mean delay per packet versus the offered traffic intensity plots are shown 
in figures 7, 9, and 11. The BIS devices delay plot shows a rising curve; the OIS devices 
(Intensive and Non-Intensive) delay plot shows a knee curve. 
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Figure 6 BIS Network Throughput Comparison Plot 
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Figure 9 Non-lntensive Network Delay Comparison Plot 
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Intensive Network Throughput Comparison Plot 



Figure 10 Intensive Network Throughput Comparison Plot 
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Figure 11 Intensive Network Delay Comparison Plot 
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In the case of the BIS devices plot, the mean delay per packet increases as the 
offered traffic intensity increases. The delay curves level out close to a traffic intensity 
of 40 Kbps per node and then start to climb linearly. This agrees with the generally 
accepted assumption that a CSMA/CD network overloads at somewhere between 30% 
and 50% of its maximum transmission speed [3]. Note that the transmission speed of 
a link (10 Mbps for Ethernet) equals the traffic intensity per node times the number 
of nodes. There are approximately 80 nodes per link in the BIS section; 80 nodes 
transmitting at 40 Kbps is 3.2 Mbps. The simulation only calculates delays per packet 
for successful packet transmissions and does not include lost packets. 

In the case of the OIS devices, all the links show a knee at a particular traffic 
intensity. The mean delay per packet decreases beyond this traffic intensity, because 
the number of completed packets decreases beyond the knee of each link. Even though 
the traffic intensity is being increased, many nodes are not given access to the channel. 
The packets that do get through have a smaller delay because the mean delays are only 
calculated for successful transmissions of the packets in the simulation. This shows 
that the links are overloaded and only a few nodes can communicate. All other nodes 
are locked out by the excessive traffic. 

AJ^ Jhr Qughput_H^ 

The throughput versus the offered traffic intensity plots are shown in figures 6, 
8, and 10. The BIS and OIS devices throughput plot shows a rising curve. 

The curves in these figures show that the throughput increases linearly with the 
offered traffic intensity per node. But the analysis of the mean delay per packet shows 
that the links are overload beyond a certain traffic intensity. This is because only a few 
nodes are able to transmit and all others cannot. The throughput beyond this knee is 
only available to a few nodes. So the throughput for each of the links should be 
determined at the traffic intensity where the maximum delay per packet occurred. 
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Link 

No of nodes 

Mean delay/packet at 
traffic intensity of 40 
kbps in msec 

Throughput at traffic 
intensity of 40 kbps in 
Mbps. 

MAC Network 

80 

1.5 

3.2 

Printer Network 

56 

0.5 

2.2 

B_2087_NI 

14 

2.0 

0.5 

RM 507 

57 

2.0 

2.3 

RM 638A 

80 

1.8 

3.4 


Table 3: Delay per packet and throughput for the BIS devices 


Link 

No of nodes 

Maximum delay/ 
packet (msec) 

Traffic intensity 
at maximum 
delay/packet 
(kbps) 

Throughput at 
maximum delay 
per packet (Mbps) 

B_1001_NI 

3 

0.004 

38 

0.11 

B_1045_NI 

4 

0.004 

38 

0.10 

B_1010_NI 

10 

23 

38 

0.46 

B_2066_NI 

4 

0.007 

38 

0.32 

RM 638B 

1 

0.002 

38 

0.81 

B_1002_NI 

1 

0.006 

38 

0.04 

B_2029_NI 

9 

23 

38 

0.4 

B_2030_NI 

9 

23 

38 

0.38 

B_1025_NI 

1 

0.006 

38 

0.04 

B_2031_I 

15 

3.0 

65 

1.0 

B_2029_I 

24 

1.5 

38 

5.0 

B_1016_I 

43 

4.0 

65 

2.4 

B_2030_I 

9 

1.0 

38 

0.3 


Table 4: Maximum delay per packet and throughput for the OIS devices 


4.5. Conclusions 

This paper has introduced the basic operating characteristics of the network 
installed at the ASRM site located in Iuka, Mississippi. An overview of the network 
topology, communication protocols, and hardware devices was presented. 

The main objective of the research was to model, simulate, and analyze the 
network to determine its performance. The two primary evaluation parameters used 
to judge the network performance were the throughput and the delay. 
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From the results obtained it can be concluded that, 

1. For the BIS devices the mean delay per packet increases as the offered traffic 
intensity increases, while for the OIS devices all the links show a knee at a 
particular traffic intensity. 

2. The throughput for all the sub-networks increases linearly with the traffic 
intensity per node. 

3. Among all the sub-networks the Intensive buildings experiences the lowest 
maximum delays. 

4. From the delay plots it can be seen that the BIS network is the most heavily 
loaded network. This can be attributed to 400 Macintosh workstations 
connected to the BIS. 

5. The buffer capacity of the bridges does appear to produce a bottleneck for the 
network. 

6. The FDDI backbone ring seems to provide ample bandwidth as was expected. 

7. The FDDI backbone causes only a negligible delay compared to the other 
delay-causing factors considered in section 4.2. 
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